

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">


<html xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
    
    <title>Database Uploads &mdash; Cogent Viewer 0.1 documentation</title>
    
    <link rel="stylesheet" href="_static/rtd.css" type="text/css" />
    <link rel="stylesheet" href="_static/pygments.css" type="text/css" />
    
    <script type="text/javascript">
      var DOCUMENTATION_OPTIONS = {
        URL_ROOT:    '',
        VERSION:     '0.1',
        COLLAPSE_INDEX: false,
        FILE_SUFFIX: '.html',
        HAS_SOURCE:  true
      };
    </script>
    <script type="text/javascript" src="_static/jquery.js"></script>
    <script type="text/javascript" src="_static/underscore.js"></script>
    <script type="text/javascript" src="_static/doctools.js"></script>
    <link rel="top" title="Cogent Viewer 0.1 documentation" href="index.html" />
    <link rel="up" title="Working with the software" href="code.html" />
    <link rel="next" title="Rest Interface" href="restInterface.html" />
    <link rel="prev" title="The Database and Model API" href="models.html" /> 
  </head>
  <body>
    <div class="related">
      <h3>Navigation</h3>
      <ul>
        <li class="right" style="margin-right: 10px">
          <a href="genindex.html" title="General Index"
             accesskey="I">index</a></li>
        <li class="right" >
          <a href="py-modindex.html" title="Python Module Index"
             >modules</a> |</li>
        <li class="right" >
          <a href="restInterface.html" title="Rest Interface"
             accesskey="N">next</a> |</li>
        <li class="right" >
          <a href="models.html" title="The Database and Model API"
             accesskey="P">previous</a> |</li>
        <li><a href="index.html">Cogent Viewer 0.1 documentation</a> &raquo;</li>
          <li><a href="code.html" accesskey="U">Working with the software</a> &raquo;</li> 
      </ul>
    </div>  

    <div class="document">
      <div class="documentwrapper">
        <div class="bodywrapper">
          <div class="body">
            
  <div class="section" id="database-uploads">
<h1>Database Uploads<a class="headerlink" href="#database-uploads" title="Permalink to this headline">¶</a></h1>
<div class="admonition warning">
<p class="first admonition-title">Warning</p>
<p class="last">The Push scripts (and webinterface) require the latest version of the
database.  Even if the server is not running the web inteface. You will need
to follow the database upgrade instructions found here
<a class="reference internal" href="install.html#upgrading-the-database"><em>Updating the Database</em></a></p>
</div>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">These scripts require a mysql database, even for testing.
Unfortunately sqlites table locking mechinism does not play
nicely when reading / writing data from several scripts.</p>
</div>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">Currently the sync is one time / one way only, (ie push from local to remote
database) This means that any changes made in either database will not be
reflected.  This would be an interesting todo for the future.</p>
</div>
<div class="section" id="overview">
<h2>Overview<a class="headerlink" href="#overview" title="Permalink to this headline">¶</a></h2>
<p>The Database Upload (Push) functionality, enables a database on a remote server to
be uploaded to a remote (central) server. Without having to resort to command
line scripting, mysql dumps etc.</p>
<p>The push function is designed to be run as a daemon process, periodically
pushing updates to the central point.</p>
<p>This functionality makes use of the REST interface (See <a class="reference internal" href="restInterface.html#rest-interface"><em>Rest Interface</em></a>.)</p>
<div class="section" id="installing-push-script">
<h3>Installing Push Script<a class="headerlink" href="#installing-push-script" title="Permalink to this headline">¶</a></h3>
<p>The Push Script currently lives in the Base <a class="reference external" href="http://code.google.com/p/cogent-house/">Cogent-house</a> software.  Follow the install
instructions at this site to install the push script.</p>
</div>
</div>
<div class="section" id="testing-configuring-the-push-script">
<h2>Testing / Configuring the push script<a class="headerlink" href="#testing-configuring-the-push-script" title="Permalink to this headline">¶</a></h2>
<p>Test the connection to the remote server by visiting the &lt;base&gt;/rest/sensortype/
url in your webbrowser (for example <a class="reference external" href="http://127.0.0.1:6543/rest/sensortype/">http://127.0.0.1:6543/rest/sensortype/</a> if
you are running the web interface via the paste server) There should be a JSON
dictionary of sensortype objects displayed.</p>
<div class="section" id="configuring-the-push-script">
<h3>Configuring the Push Script<a class="headerlink" href="#configuring-the-push-script" title="Permalink to this headline">¶</a></h3>
<p>Modify the synchronise.conf file (base/push/synchronise.conf) with the details
of the remote rest server you visited above.</p>
</div>
<div class="section" id="running-the-push-script">
<h3>Running the Push Script<a class="headerlink" href="#running-the-push-script" title="Permalink to this headline">¶</a></h3>
<p>To run the push script:</p>
<div class="highlight-python"><pre>python RestPusher.py</pre>
</div>
<p>The script should output logging information to a logfile.</p>
</div>
<div class="section" id="running-the-push-script-as-a-daemon">
<h3>Running the Push Script as a Daemon<a class="headerlink" href="#running-the-push-script-as-a-daemon" title="Permalink to this headline">¶</a></h3>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">Write code / Docs for this</p>
</div>
</div>
</div>
<div class="section" id="algorithm-overview">
<h2>Algorithm Overview<a class="headerlink" href="#algorithm-overview" title="Permalink to this headline">¶</a></h2>
<p>Each Iteration of the <em>sync</em> function does the following:</p>
<ol class="arabic simple">
<li>Read the Configuration Files</li>
<li>For Each Server that needs to be updated<ol class="arabic">
<li>Ensure all local nodes are in the remote table</li>
<li>Map <em>Active</em> Deployments</li>
<li>Map <em>Active</em> Houses</li>
<li>Synchronise the Location Table</li>
<li>Update Nodes with new locations</li>
<li>Upload the Relevant Node States</li>
<li>Upload the Relevant Readings</li>
<li>Update Configuration Files with last transmitted sample</li>
</ol>
</li>
</ol>
<p>These steps are described in detail below:</p>
<div class="section" id="synchronising-nodes">
<h3>Synchronising Nodes<a class="headerlink" href="#synchronising-nodes" title="Permalink to this headline">¶</a></h3>
<p>This method uses SQLA to synchronise the node tables between the local and remote servers.
As nodeIds should be static (ie node 69 will allways be node 69) then this function can make use of a simplistic approach.</p>
<ol class="arabic simple">
<li>For each local node, where the node id is not on the remote server:
#. Create a new node with this Id
#. Create a new set of sensors linked to this node</li>
</ol>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">Currently, we only create new sensors when adding a node.  At some point we may need to update the code to deal with this</p>
</div>
</div>
<div class="section" id="mapping-deployments-and-houses">
<h3>Mapping Deployments and Houses<a class="headerlink" href="#mapping-deployments-and-houses" title="Permalink to this headline">¶</a></h3>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p>We Assume that the following parameters are unique</p>
<ol class="last arabic simple">
<li>Deployment.name</li>
<li>House.address</li>
</ol>
</div>
<p>To reduce the amount of network traffic, we only map <em>Active</em> Deployments /
Houses, where an active house is defines as one that:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="nb">object</span><span class="o">.</span><span class="n">endDate</span> <span class="o">==</span> <span class="bp">None</span>
<span class="nb">object</span><span class="o">.</span><span class="n">endDate</span> <span class="o">&gt;=</span> <span class="n">lastUpdate</span>
</pre></div>
</div>
<p>These database items are fetched from the local database, and mapped to their equivelent on the remote database.
If no such item exists on the remote database then</p>
<p>The following queries are first run on the local database</p>
<ol class="arabic">
<li><p class="first">Get Deployments / Houses to be updated</p>
<p>Given the date of the last update, we fetch all deployments and houses where
the finishdate is either None, or After the last update.  This should reduce
the amount of locations mapped to a minimum.</p>
<p>If for some reason there are no deployments, then we get all active &#8220;houses&#8221;
using the same parameters on the house table.</p>
</li>
<li><p class="first">Get Locations to be Updated</p>
<p>Based on the houseID&#8217;s of those houses that need are current.  We fetch a set
of locations from the local database.</p>
</li>
<li><p class="first">Get Rooms to be Updated</p>
<p>Now we have the location Id&#8217;s, it is possible to get a list of rooms that will need updating</p>
</li>
<li><p class="first">Create new objects</p>
</li>
</ol>
<table class="docutils citation" frame="void" id="locations" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label">[locations]</td><td></td></tr>
</tbody>
</table>
<table class="docutils citation" frame="void" id="local" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label">[local]</td><td></td></tr>
</tbody>
</table>
</div>
</div>
</div>


          </div>
        </div>
      </div>
      <div class="sphinxsidebar">
        <div class="sphinxsidebarwrapper">
  <h3><a href="index.html">Table Of Contents</a></h3>
  <ul>
<li><a class="reference internal" href="#">Database Uploads</a><ul>
<li><a class="reference internal" href="#overview">Overview</a><ul>
<li><a class="reference internal" href="#installing-push-script">Installing Push Script</a></li>
</ul>
</li>
<li><a class="reference internal" href="#testing-configuring-the-push-script">Testing / Configuring the push script</a><ul>
<li><a class="reference internal" href="#configuring-the-push-script">Configuring the Push Script</a></li>
<li><a class="reference internal" href="#running-the-push-script">Running the Push Script</a></li>
<li><a class="reference internal" href="#running-the-push-script-as-a-daemon">Running the Push Script as a Daemon</a></li>
</ul>
</li>
<li><a class="reference internal" href="#algorithm-overview">Algorithm Overview</a><ul>
<li><a class="reference internal" href="#synchronising-nodes">Synchronising Nodes</a></li>
<li><a class="reference internal" href="#mapping-deployments-and-houses">Mapping Deployments and Houses</a></li>
</ul>
</li>
</ul>
</li>
</ul>

  <h4>Previous topic</h4>
  <p class="topless"><a href="models.html"
                        title="previous chapter">The Database and Model API</a></p>
  <h4>Next topic</h4>
  <p class="topless"><a href="restInterface.html"
                        title="next chapter">Rest Interface</a></p>
  <h3>This Page</h3>
  <ul class="this-page-menu">
    <li><a href="_sources/push.txt"
           rel="nofollow">Show Source</a></li>
  </ul>
<div id="searchbox" style="display: none">
  <h3>Quick search</h3>
    <form class="search" action="search.html" method="get">
      <input type="text" name="q" />
      <input type="submit" value="Go" />
      <input type="hidden" name="check_keywords" value="yes" />
      <input type="hidden" name="area" value="default" />
    </form>
    <p class="searchtip" style="font-size: 90%">
    Enter search terms or a module, class or function name.
    </p>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
        </div>
      </div>
      <div class="clearer"></div>
    </div>
    <div class="related">
      <h3>Navigation</h3>
      <ul>
        <li class="right" style="margin-right: 10px">
          <a href="genindex.html" title="General Index"
             >index</a></li>
        <li class="right" >
          <a href="py-modindex.html" title="Python Module Index"
             >modules</a> |</li>
        <li class="right" >
          <a href="restInterface.html" title="Rest Interface"
             >next</a> |</li>
        <li class="right" >
          <a href="models.html" title="The Database and Model API"
             >previous</a> |</li>
        <li><a href="index.html">Cogent Viewer 0.1 documentation</a> &raquo;</li>
          <li><a href="code.html" >Working with the software</a> &raquo;</li> 
      </ul>
    </div>
    <div class="footer">
        &copy; Copyright 2011, Daniel Goldsmith.
      Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 1.1.3.
    </div>
  </body>
</html>